Understanding different appearances of an object is an elementary problem in various fields. Since acquisition conditions vary (e.g., pose, illumination, acquisition system), the set of possible observations on a particular object is immense; therefore, the complicated task of characterizing and determining the relation between a pair of observations is crucial whether one is after the differences themselves (e.g., target tracking) or whether the interest is restricted to determining if two observations are of the same object (e.g., face recognition).
Image registration, and in general the problem of estimating transformations of observed objects, has been intensively studied for several decades. Results of this research have been applied in various fields such as remote sensing, medical imaging and computer vision, and to problems such as super-resolution, video compression, and many more.
The vast majority of studies focus on geometric-only registration; that is, alignment of the domain (coordinates) of images. Correspondingly, the term “registration” is commonly associated with geometry, which indeed is a challenging problem in itself. Intuitively, it seems that the simplest way to relate a pair of geometrically deformed images is by associating ordered sequences of some salient features, that can be located on both images (feature-based methods). These are usually selected to be line or point features, and thus, much of the intensity information contained in the image is discarded. To make this approach feasible, the problem of finding the correspondence between the features on both images must be solved first. Therefore, this approach may only be used when the features are easily detectable, the deformation is close enough to the identity, the number of features is relatively small (yet large enough to allow for a meaningful estimation), and there is a strong contextual evidence to guide the solution to the correspondence problem. Yet, in many problems the feature points are not easily identifiable and their number may be large, in which case the correspondence problem rapidly becomes very difficult to solve.
Featureless methods (area methods) avoid the correspondence problem by directly employing the intensity information of the image in order to establish an estimate of the geometric deformation, without identifying any features. Among such methods are those based on “correlation-like”, Fourier, mutual information and optical flow principles. Typically, these methods lead to the utilization of some iterative optimization procedure in order to estimate the deformation. As such, they are applicable only when the deformations are small, which lowers the risk of obtaining a local minimum as the solution. Explicit solution methods, which do not employ the minimization of a penalty function of some sort, are commonly restricted to a relatively small family of transformations. For example, there are explicit methods in the cases of translation only, rotation only, or scale only, but the solution becomes (partially) optimization based once combinations of translation, rotation and scaling are introduced.
On the other hand, various recent studies have focused on radiometry-only registration; that is, alignment of the range (values/intensities) of geometrically-aligned images. This problem is much simpler since images are geometrically pre-aligned, and thus pointwise correspondence inherently exists. Thus, straightforward methods may be used to combine images captured in different optical settings into a single image of high dynamic range (HDR).
Nonetheless, it seems that the problem of deriving a registration procedure in the presence of combined radiometric and geometric variations (i.e., both domain and range) has received much less attention. In this case, employing geometric registration methods without taking the radiometric variation into account is clearly far from optimal: featureless geometric registration methods are extremely sensitive to intensity variations and typically fail in the presence of some non-negligible radiometric transformation relating the intensities of the observations. This therefore leaves feature-based methods as a sole option, since they are less sensitive to radiometric variations. These methods, however, are not always applicable, as mentioned above. For example, in medical imaging, cross-modality geometric registration procedures are prone to failure due to significant, difficult-to-model, differences in pixel (voxel) intensities; this has led researchers to the employment of various computationally demanding variational methods for performing the registration. In the geometric registration of optical imaging, it is customary to evade the radiometric effects by using “radiometry-invariant” procedures (usually feature-based, as previously mentioned). The physical understandings and invariancy principles of the color constancy framework have also been utilized in attempt to minimize the effects of such radiometric variations (these, however, usually assume linear radiometric effects). Another approach, that relies on the principals of the aforementioned radiometry-only registration, overcomes radiometric phenomena in the registration of optical images (and, in fact, benefits from it) by using special optical apparatus (spatially varying filters attached to the camera).